Depth dependent filtering of image signal

ABSTRACT

A method and apparatus for rendering of image data for a multi-view display, such as image data for a lenticular auto-stereoscopic display, is disclosed. The method comprises the steps of receiving an image signal representing a first image, the first image comprising 3D image data, and spatially filtering the first image signal to provide a second image signal. The second image signal represents a second image, the spatial filtering being, e.g., a low-pass filter, a high-pass filter or a combination of a low-pass and a high-pass filter. A strength of the spatial filter is determined by a reference depth of the first image and a depth of an image element of the first image. The second image is sampled to a plurality of sub-images, each sub-image being associated with a view direction of the image.

This application is a continuation of U.S. patent application Ser. No.12/095,176 filed on 26 Aug. 2008, which application issued as U.S. Pat.No. 8,624,964 on 7 Jan. 2014.

The invention relates to a method of rendering image data for amulti-view display. In particular the invention relates to a method ofrendering image data for a multi-view display by means of a depthdependent spatial filter. The invention further relates to a multi-viewdisplay, to a signal rendering system and to computer readable code forimplementing the method.

A multi-view display is a display capable of presenting to a viewer,different images depending upon the view direction, so that an object inan image may be viewed from different angles. An example of a multi-viewdisplay is an auto-stereoscopic display capable of presenting a viewer'sleft eye with a different image than the right eye. Various multi-viewdisplay technologies exist, one such technology is lenticular based. Alenticular display is a parallax 3D display capable of showing multipleimages for different horizontal viewing directions. This way, the viewercan experience, e.g., motion parallax and stereoscopic cues.

One problem relating to multi-view displays is that images for differentview-directions may overlap and thereby giving rise to ghost images, orcross-talk between images. Another problem relates to that the number ofview-directions may be relatively small, typically eight or nine whichmay give rise to aliasing effects in some view-directions.

The published US patent application US 2003/0117489 discloses a threedimensional display and method of reducing crosstalk between left andright eye images of a 3D auto-stereoscopic display. The disclosed methodof reducing crosstalk is based on adding a base level of grey to everypixel of both the left and right images so as to raise the backgroundgrey level.

The inventor of the present invention has appreciated that an improvedmethod of rendering image data is of benefit, and has in consequencedevised the present invention.

The present invention seeks to provide improved means for renderingimage data for a multi-view display, and it may be seen as an object ofthe invention to provide an effective filtering technique thatameliorates the perceived image quality of a viewer, or user, of amulti-view display. Preferably, the invention alleviates, mitigates oreliminates one or more of the above or other disadvantages singly or inany combination.

According to a first aspect of the present invention there is provided,a method of rendering image data for a multi-view display, the methodthe comprising steps of:

receiving an image signal representing a first image, the first imagecomprising 3D image data,

spatially filtering the first image signal to provide a second imagesignal, the second image signal representing a second image, the spatialfiltering comprising a mapping between an image element of the firstimage and an image element of the second image, a strength of thespatial filter is determined by a reference depth of the first image anda depth of an image element of the first image,

sampling the second image to a plurality of sub-images, each sub-imagebeing associated with a view direction of the image.

In a multi-view display, the image data is typically rendered for properpresentation. The rendering may be needed since the image may be basedon 2D image data projected to the viewer in such a way that the viewerperceives a spatial, or 3D, dimension of the image. For eachview-direction of an image, a sub-image of the image as seen from thatview-direction is generated, and the sub-images are projected into theassociated view-direction.

The rendering process typically comprises several operations or steps,e.g. depending upon the input format of the image data, the displayapparatus, the type of image data, etc. Image data of a first image isprovided in a first step. This first step need not be a first step ofthe entire rendering process. The first image is typically in a formatincluding image plus depth data, or an associated depth map may beprovided with the image data, so that the 3D image data may bedetermined.

The inventor had the insight that spatial filtering for improving theperceived image quality, especially in terms of crosstalk and aliasingeffect, is performed in the output domain, i.e. it is performed at arendering stage where an input image has already been sampled, at leastto some degree, for multi-view display. By spatially filtering the firstimage signal to provide a second image and the second image beingsampled to a plurality of sub-images for multi-view, artefacts, such ascrosstalk and aliasing effects, are dealt with in the input domain on asingle image instead of in the output domain on a plurality of images,thereby dealing with artefacts in an efficient way.

While filtering a single image in the input domain rather than themultitude of images in the output domain, may be less perfect than thefull-blown filtering of the multitude of images in the output domain,most artefacts may still be avoided or diminished, and a low-costalternative in terms of processing power and time may thereby beprovided.

Further advantages of the invention according to the first aspectinclude easy implementation in the rendering pipeline of images formulti-view display. The invention may be implemented in a separatepipeline step before the actual multi-view rendering, allowing for amore pipelined parallel implementation.

Furthermore, the method is effectively dealing with reduction ofartefacts, such as crosstalk and aliasing artefacts, thereby renderingpre-processing or post-processing to further remove or diminishcrosstalk or aliasing artefacts unnecessary.

Advantageous optional features include band-pass filtering done bylow-pass filtering, high-pass filtering and/or a combination of the two,which are well-known band-pass filtering techniques that may beimplemented in variety of ways, thereby ensuring robust and versatileimplementation. In low-pass filtering, frequencies higher than theNyquist frequency may be removed, whereas high-pass filtering amplifieshigh frequencies, e.g. the frequencies below the Nyquist frequency.

Additional advantageous optional features include mapping of the imageelement of the first image into a set of image elements of the secondimage and determining the strength of the spatial filter as a size ofthe set of image elements of the second image, such as a radius orextent of a distribution filter of the set of image elements. Thereby,it is ensured that objects near the reference plane are not greatlyaffected by the spatial filtering, whereas objects further away from thereference plane are affected by the spatial filtering.

Additional advantageous optional features include updating an imageelement of the second image with a visibility factor, whereby problemsrelating to mixing of foreground and background objects may be counteredin an effective way. Such problems may arise when a spatial filteredimage is rendered for a shifted viewpoint.

Additional advantageous optional features include updating the depth ofthe image elements of the second image, whereby an improved handling ofviewpoint changes may be provided. The depth is updated by setting thedepth of the image element of the second image to a value between thedepth of the image element of the first image and the depth of the imageelement of the second image. In this way, when an image element of thesecond image would substantially be composed of foreground and only alittle of background, the depth may be set to a value substantiallytowards the depth of the foreground, providing a gradual depthtransition that softens the depth edge. In an embodiment the depth valuemay be set to the maximum of the depth of the image element of the firstimage and the depth of the image element of the second image.

Additional advantageous optional features include applying the spatialfilter so that the image element of the first image and the set of imageelements of the second image are aligned along a horizontal line of thefirst image, whereby effects of the coarse sampling in the viewdirection and crosstalk may effectively be countered for a multi-viewdisplay projecting the difference views in a plurality of horizontallyorientated directions.

An additional advantageous optional feature includes use of the 2.5Dvideo image format for the first image signal, since this is a standardand widely used format.

According to a second aspect of the invention is provided a multi-viewdisplay device comprising:

a display panel comprising an array of display elements, the displayelements being arranged in groups, each group being associated with aview direction of an image,

an optical element for directing light emitted from the display panel,so that light emitting from a group of display elements is directed intoan angular distribution associated with the view direction of the group,

an input module for receiving a first image signal,

a rendering module for spatially filtering the first image signal toprovide a second image signal, the second image signal representing asecond image, the spatial filtering comprising a mapping between animage element of the first image and an image element of the secondimage, a strength of the spatial filter is determined by a referencedepth of the first image and a depth of an image element of the firstimage,

an output module for sampling the second image to a plurality ofsub-images, each sub-image being associated with a view direction of theimage.

Since the display device comprises a multi-view display device enhancedwith the rendering method of the first aspect, it is an advantage of thepresent invention that the multi-view display device may be either adisplay device that initially includes the functionality according tothe first aspect of the invention, or a display device to which thefunctionality according to the first aspect of the invention issubsequently added.

The input module, the rendering module and the output module may beprovided as a signal rendering system according to the third aspect ofthe invention.

According to a fourth aspect of the invention, a computer readable codefor implementing the method according to the first aspect is provided.

In general the various aspects of the invention may be combined andcoupled in any way possible within the scope of the invention. These andother aspects, features and/or advantages of the invention will beapparent from and elucidated with reference to the embodiments describedhereinafter.

Embodiments of the invention will be described, by way of example only,with reference to the drawings, in which

FIG. 1 illustrates the principle of a 3D lenticular display,

FIGS. 2A and 2B show sketches of a lenticular display in top view,

FIG. 3 illustrates crosstalk between sub-images from neighboring views,

FIG. 4 illustrates an example of blurring of an image as a result of acamera focused at a particular depth,

FIGS. 5A-5C illustrate an embodiment of a mapping of a first image to asecond image in a spatial filtering process,

FIGS. 6A and 6B illustrate an example of a 2D input image withassociated depth map,

FIGS. 7A and 7B illustrate a depth dependent spatial filtering of theinput image and a shifted view of the same image (or scene),

FIGS. 8A and 8B illustrate a depth dependent spatial filtering of theinput image where a visibility factor has been applied and a shiftedview of the filtered image,

FIGS. 9A and 9B illustrate a depth dependent spatial filtering of theinput image where an adjusted visibility factor has been applied and ashifted view of the filtered image,

FIGS. 10A and 10B illustrate a spatially filtered depth map and depthdependent spatial filtering of the input image with the filtered depthmap,

FIGS. 11A and 11B illustrate a depth dependent spatial filtering of theinput image for a 1D situation and a shifted view of the filtered image,

FIGS. 12A-12F illustrate aspects concerning the application of high-passspatial filtering.

FIGS. 1 and 2 illustrate embodiments of a multi-view display, namelysketches of a 3D lenticular display as seen from the side (FIG. 1) andin top view (FIG. 2).

FIG. 1 illustrates the principle of a 3D lenticular display. Alenticular display is based on an LCD panel display 1, in front of whichlenses 2 are attached. The lenses accommodate that for a specificviewing angle φ, the viewer 3 only sees a subset of the pixels of theunderlying LCD. If appropriate values are set to the subset of pixelsassociated to the various viewing directions, the viewer will seedifferent images from different viewing directions. So the viewer 3 seesa center view of the image, whereas the viewer would see a side view ofthe image from the view-angle denoted 6.

Each lens covers a number of pixels 4, 5 and projects them out, asillustrated by the number of pixels denoted 7. The viewer sees onesubset of pixels 4 with the right eye and another subset of pixels 5with the left eye. A 3D experience is thereby obtained.

FIG. 2A shows a sketch of lenticular display in top view. The displaycomprises an array of display elements 20 or pixels, such as aconventional LC matrix display panel, where the pixels are arranged ingroups, each group being associated with a view direction of an image.Each group of pixel constitutes a sub-image, each sub-image beingassociated with a view direction. An optical element, i.e. the lenses,direct light emitted from the pixels so that light emitting from a groupof pixels is directed into an angular distribution associated with theview direction of the group, thereby providing separate images to aviewer's eyes.

The lenticular lenses are in the illustrated embodiment arranged at aslight angle or slanted with respect to the columns of the pixels, sothat their main longitudinal axis is at an angle with respect to thecolumn direction of the display elements. In this configuration theviewer will see the points sampled along a direction 22 of the lens. Ina nine-view display, nine images, one for each view direction, areconcurrently computed and shown on the group of pixels associated with asub-image. When a pixel is lit, the entire lens above the pixel isilluminated 21—this is shown in FIG. 2B—so that for a specific viewdirection it is the entire lens above the pixel that is seen emittingthe color of that pixel.

FIGS. 1 and 2 describe a LCD-lenticular display; it is however to beunderstood that the invention is not limited to this type of display.For example, the invention may be applied with such displays asbarrier-type displays, and the matrix display panel may be other than anLC panel, such as other forms of spatial light modulators, or othertypes of display panels such as electroluminescent or plasma panels.

The visibility of sub-images from neighboring views from a singleviewing direction may cause artefacts such as crosstalk. This isillustrated in FIG. 3 showing the visible light, I, as a function of theview angle for a 4⅔-display, i.e. for a display where each lens cover 4⅔pixels in the horizontal direction. It is seen that the angulardistributions 30-34 from different sub-images overlap. The imageperceived by a viewer is the sum of the light from each angulardistribution and it may be seen that for this particular example, threesub-images contribute to the perceived image of each viewing direction.

The inventor of the present invention has appreciated that byappropriate spatial filtering problems relating to crosstalk, to ghostimaging and aliasing may be removed or at least diminished. Furthermore,by spatially filtering the input image before the image is rendered formulti-view display, only a single image needs to be filtered (andpossible a depth map in accordance with certain embodiments), therebyproviding an efficient way of handling spatial filtering of multi-viewimage data.

Depth dependent spatial filtering is done to counter crosstalk and/oraliasing effects. However, the depth dependent spatial filtering on aninput image which is subsequently rendered for different viewpoints mayintroduce new artefacts by the rendering. For example, artefactsrelating to foreground and background objects mix for the renderedimages with shifted viewpoint, thereby diminishing the perceived imagequality of the 3D image at the different viewpoints.

In order to provide a 3D image with high perceived image quality, thedepth dependent filtering of the image may be such that a blurring ofthe image is consistent with a blur introduced by a camera focused at aparticular depth, this is illustrated in FIG. 4. The Figure shows ascene from the TV-series Star Trek Enterprise, showing two actors 41, 42in front of a blurry background 44. The actor denoted 42 is also out offocus and as a consequence of this, blurred. Looking at the shoulder ofthe actor denoted 41 it is clear that background does not blur overforeground objects, as a sharp outline 43 of the shoulder is seen. Theshoulder outline 45 of the actor denoted 42 is, however, blurred,showing that foreground objects do blur over background object. In animage rendering process, following these rules of what blurs over whatin an image leads to an increased perceived spatial dimension in a 3Dimage.

The band-pass filter is typically a low-pass or a high-pass filter. Thelow-pass filter mitigates problems, typically alias problems, related tosampling the intensity function into a low number of sub-images, such aseight or nine, depending upon the number of views of the display. Thehigh-pass filter mitigates problems relating to crosstalk imposing blurin the view direction. A combination of high-pass filtering and low-passfiltering may be performed to optimize the perceived image quality, orthe filters may be applied separately.

FIGS. 5A-5C illustrate an embodiment of a mapping of a first image to asecond image in a spatial filtering process.

Firstly, an image signal representing a first image comprising 3D imagedata is received or provided. The 3D image data may be represented inany suitable coordinate representation. In a typical coordinaterepresentation, the image is described in terms of a spatial coordinateset referring to a position in an image plane, and a depth of the imagein a direction perpendicular to the image plane. It is, however, to beunderstood that alternative coordinate representations may beenvisioned.

The filtering may be input-driven, and for each input pixel, the inputpixel also being referred to as the source element, the difference indepth between the source element and a reference depth is determined.The reference depth being set to the depth layer in the image which isin focus, or which should remain in focus. The depth difference is thenused as a measure for the strength of the spatial filter. The strengthof the filter may, in an embodiment, be the number of pixels affected bythe intensity of the source element, i.e. as the size of the set ofimage elements of the second image. The size of the set of imageelements may be the radius of a distribution filter, distributing theintensity of the source element to the set of destination elements. Inthe following, source element and destination element are alternativelyreferred to as source pixel and destination pixel, respectively.

FIG. 5A schematically illustrates the principle of using the differencebetween the depth of a given pixel and a reference depth, d-dref, as ameasure of the radius, r, of a distribution filter. The same isillustrated in FIGS. 5B and 5C, but concretized with a section of amatrix display panel, such as a LCD display, comprising a number ofimage elements or pixels 51. A reference depth is set for the entireimage, and for each pixel a depth of the pixel is determined. Theintensity value of a pixel 52 is distributed in a set of pixels in thesecond image 53, the second image being an updated version of the firstimage 50 where, for all pixels 52, the intensity of the pixel isdistributed to a set of pixel elements surrounding the pixel and thesize of the set, or the radius, r, of the area 55 of the affected pixel,is determined from d-dref. The intensity of a pixel in the set of pixels55 may be determined as Ip:=Ip+f(r)*Iq, where Ip is the intensity of thepixel, i.e. the output intensity that is accumulating, f(r) is thedistribution function and Iq is the intensity of the source pixel at adistance r of the destination. The distribution filter may be a cubicb-spline.

For areas where the depth values are near the reference depth, theradius of the distribution filter is small, so destination pixels onlyreceive contribution from the corresponding source pixel. For areaswhere the depth is much different from the reference value, source pixelintensities are distributed over large areas and the mix, resulting in ablur.

To generate a blur that is consistent with the blur introduced bycameras focused on a particular depth, a visibility factor, v, ismultiplied by the distribution function, so that Ip:=Ip+f(r)*Iq*v. Thevisibility factor equals zero when the destination pixel is much closerto the viewpoint than the source pixel, thereby ensuring that backgrounddoes not blur over foreground. The visibility factor equals one when thesource pixel is much closer to the viewpoint than the destination pixel,and has a gradual transition between the two values. The distancesbetween the source pixel, the destination pixel and the viewpoint may beevaluated from the spatial coordinates of the source pixel, thedestination pixel and the viewpoint, for example by comparing thedistances between the source pixel and the viewpoint and between thedestination pixel and the viewpoint to one or more minimum distances soas to determined when the source pixel is much closer than thedestination pixel to the viewpoint, and vice versa. The visibilityfactor has the effect that destination colors have to be normalizedaccording to the summation of weights, since the sum of weights cannotbe held constant beforehand.

In the following embodiments relating to depth-dependent blur areaddressed. A depth-dependent spatial filter can nevertheless be used toboth depth-dependent blurring (low-pass filtering) and to depthdependent sharpening (high-pass filtering) which is discussed inconnection with FIG. 12.

FIGS. 6 to 11 illustrate the effect of applying a depth filter inaccordance with the present invention on a scene from the game Quake™.

FIG. 6A illustrates the scene as used in the game, and FIG. 6Billustrates the depth map of the image (or scene). In the depth map, thegrey-scale corresponds to disparity, so that bright objects are closerthan dark objects. The reference depth is set to the pillar 60 in themiddle. In the following images, the image of FIG. 6A is the first imageto be mapped into a second image by a spatial filtering according to thepresent invention. The image of FIG. 6A in combination with the depthmap of FIG. 6B is hereafter referred to as the source image. The secondimage may hereafter be referred to as the destination image.

FIGS. 7A and 7B illustrate the depth-dependent blur without thevisibility factor. FIG. 7A shows a blurring of the source image withoutthe visibility factor, i.e. a destination image obtained without thevisibility factor. The image of FIG. 7B is derived from the destinationimage of FIG. 7A for a shifted viewpoint using the depth map shown inFIG. 6B.

It is seen in FIG. 7A that the background is blurring over theforeground. This may e.g. be seen by the white background area whichblurs over the pillar 70. This blurring has an effect for the shiftedviewpoint (FIG. 7B), since some white area 71 may be seen even though itshould not be visible from the specific view-angle. Blurring ofbackground over foreground results in a halo-effect counteracting theocclusion.

FIGS. 8A and 8B illustrate the depth-dependent blur with the visibilityfactor. FIG. 8A shows a blurring of the source image with the visibilityfactor, i.e. a destination image obtained with the visibility factor.The image of FIG. 8B is derived from the destination image of FIG. 8Afor a shifted viewpoint, similar to FIG. 7B.

Both in the destination image (FIG. 8A) and the shifted viewpoint image(FIG. 8B) are the artefacts discussed in connection with FIGS. 7A and 7Bremoved. A sharp edge is obtained for the left side of the pillar 80,and the pillar occludes the white area in the shifted view 81.

However, the halo-artefacts remain for silhouettes where foregroundblurs over background. The de-occlusion that occurs with the object atthe lower left corner 84 is not just a repetition of a background color,but of a color which for a large part is made up of foreground color,i.e. a semi-transparency is introduced.

In a situation where additional viewpoints of an image are rendered fromimage plus depth information, different solutions exist for diminishingor even removing the halo-effects.

In an embodiment, the visibility factor is modified so that sourcepixels only contribute to destination pixels of similar depth. Theresult of such a filtering is shown in FIGS. 9A and 9B. It may be seenthat the halo-effect is removed, but the sharp edge of the foregroundobject 90, 91 remains. Such sharp silhouette edges may result in doubleimages due to crosstalk, even though the interior of the object isblurred.

In another embodiment, halo-effects are countered by filtering the depthmap. The halo-effects 84 as seen in the lower left corner of FIG. 8B arelargely due to foreground color being repeated, because the pixels thatoriginally contained background colors only contain a lot of foregroundcolor after the blurring. The halo-artefacts are enlarged when shiftedviews are computed, because a color which consists for a large part offoreground color is now used to fill in a de-occlusion area.

A solution, which at least reduces the artefacts considerably, is toalso filter the depth map itself, thereby ensuring that the artefactsare not enlarged as much by the rendering.

Any destination pixel to which a foreground color is distributed, shouldalso have foreground depth, thereby avoiding the use of such a pixel inthe multi-view rendering to fill in de-occlusion areas. This can be doneby applying a depth-dependent morphological filter: when a source pixelis distributed to a destination pixel, the depth of the destinationpixel is set to the maximum of the depth of the source pixel and theprevious depth of that destination pixel. This naturally follows thevisibility criterion: depth information from background objects does notchange depth information of foreground objects (which for example willkeep the depth transitions of for example the pillar to its backgroundsharp, both in color and in depth). In general, the updating of thedepth map may be done by instead of setting the depth of the destinationpixel to the maximum value as mentioned above, to set the depth of thedestination pixel to a value between the depth of the source pixel andthe depth of the destination pixel.

In a situation where the image filter blurs foreground over background,the depth map is updated with the foreground depth to extend theforeground object. The result is shown in FIG. 10A, showing the updateddepth map. Comparing the depth map of FIG. 6B and the depth map of FIG.10A, the dilation of the foreground objects is clear (the object in thelower left corner 101, but also in the background to the right of thepillar 100).

Using this filtered depth map, along with the filtered image from FIG.8A results in the alternate view shown in FIG. 10B. Some halos 102 arestill visible due to the semi-transparency of the blurred edges, whichare now rendered at foreground depth, and also due to de-occlusions nowbeing filled with color information originating from quite far from theedge, but by far not as severe as those depicted in FIG. 8B.

The spatial filtering as discussed in connection with FIGS. 5 to 10 is a2D filtering in the sense that the set of destination pixels is a set ofpixels in an image plane. Such 2D filtering may be necessary in order tomimic the out-of-focus blur of a real camera, and thereby improve theperceived image quality of a viewer. However, to counter effects ofcoarse sampling in the view direction as may be present on multi-viewdisplay devices, as well as crosstalk, a horizontal filter may suffice.In a horizontal filter, or a 1D filter, instead of the set ofdestination pixels being comprised within an area 55 as shown in FIG.5C, the set of destination pixels extend along the horizontal directionon both sides of the source pixel. An example of an image and an imagewith a shifted viewpoint is shown in FIG. 11 for 1D horizontal depthdependent spatial filtering. As can be seen when comparing FIG. 11A toFIG. 6A, the horizontal blur has been applied. FIG. 11B shows thesituation with a shifted viewpoint for a case where the depth map hasbeen filtered, as in connection with FIGS. 10A and 10B. Also in the 1Dsituation large halo artefacts are prevented from appearing.

In horizontal filtering vertical halo-effects are avoided for shiftedviewpoints. An example of a vertical halo-effect that is avoided in thissituation may be seen by comparing the top of the pillar 110 in FIG. 11Band FIG. 7B. In FIG. 7B vertical halo-effects are introduced by theshifting of viewpoint.

A high-pass filtering is typically applied in order to pre-compensateblurring of an image introduced later on, e.g. in connection with themulti-view rendering or sampling of the image.

FIG. 12A schematically illustrates an input image with an edge.Multi-view rendering will shift this image, according to depth, so asecond view will be a shifted version of the image (assuming constantdepth in that area in this case). This is shown in FIG. 12B. On amulti-view display exhibiting crosstalk a viewer will not purely see oneview (say view 1), but see a mix of the view one should see, and theneighboring views. As an example, FIG. 12C illustrates the situationwhere ⅛th of a neighboring view is seen, thus FIG. 12C illustrates thecombination of ⅛th of a neighboring view (FIG. 12B) and ⅞th of the viewitself (FIG. 12A). The edge is split over 2 smaller steps, i.e. the edgeis blurred.

To counter this, the input image can be high-pass filtered, usuallyresulting in some overshoot before and after the edge, making the edge“higher”. This is drawn schematically in FIG. 12D. FIG. 12E shows theshifted version of FIG. 12D, and FIG. 12E illustrates a situation wherecrosstalk has been introduced, i.e. FIG. 12F is the combination of FIG.12E (the shifted view) and FIG. 12D (the original view). As shown, theedge is still sharp, despite the crosstalk.

For high-pass filtering areas which have a depth similar to thereference depth no or only little sharpening occurs, as the differencebetween the reference depth and the depth of the area increases, theradius, or extent, of the area affected by the sharpening increases,matching the distance between edges in neighboring views.

In an embodiment, the signal including the image data to be presented tothe viewer is inputted into an input module, as a first image signal.The depth dependent spatial filtering of the first image to provide asecond image is conducted at a rendering module, the rendering moduletypically being a processor unit. The input module, rendering module andoutput module, need not, but may, be separate entities.

The rendering module may also apply additional rendering functions tothe image data, e.g. the image data may be properly scaled to the viewresolution, colors may be adjusted, etc. The rendering of the imagesignal may be done separately for different color components and theview-dependent intensity function may be determined for at least onecolor component of the image, and the band-pass filtering applied to theat least one color component of the image. For example, since in anRGB-signal the green component is the most luminous component, thespatial filtering may in an embodiment only be applied for the greencomponent.

The invention can be implemented in any suitable form includinghardware, software, firmware or any combination of these. The inventionor some features of the invention can be implemented as computersoftware running on one or more data processors and/or digital signalprocessors. The elements and components of an embodiment of theinvention may be physically, functionally and logically implemented inany suitable way. Indeed, the functionality may be implemented in asingle unit, in a plurality of units or as part of other functionalunits. As such, the invention may be implemented in a single unit, ormay be physically and functionally distributed between different unitsand processors.

Although the present invention has been described in connection withpreferred embodiments, it is not intended to be limited to the specificform set forth herein. Rather, the scope of the present invention islimited only by the accompanying claims.

In this section, certain specific details of the disclosed embodimentare set forth for purposes of explanation rather than limitation, so asto provide a clear and thorough understanding of the present invention.However, it should be understood readily by those skilled in this art,that the present invention may be practiced in other embodiments whichdo not conform exactly to the details set forth herein, withoutdeparting significantly from the spirit and scope of this disclosure.Further, in this context, and for the purposes of brevity and clarity,detailed descriptions of well-known apparatus, circuits and methodologyhave been omitted so as to avoid unnecessary detail and possibleconfusion.

Reference signs are included in the claims, however the inclusion of thereference signs is only for clarity reasons and should not be construedas limiting the scope of the claims.

I claim:
 1. A method of rendering image data for effecting theproduction of sub-images by a multi-view display, the method comprisingthe steps of: receiving a first image signal representing a first imagecomprising 3D image data not yet rendered for multi-view display;spatially filtering the first image signal to provide a second imagesignal, the second image signal representing a second image, the spatialfiltering comprising mapping an image element of the first image to atleast one image element of the second image, where a strength of thespatial filter is dependent on a depth of the image element of the firstimage relative to a reference depth of the first image; sampling thesecond image signal to define a plurality of said sub-images, eachsub-image being associated with a different respective view direction ofthe image; and displaying said plurality of said sub-images on themulti-view display, wherein a visibility factor is applied to the atleast one image element of the second image, the visibility factor beingdetermined from a distance between a viewpoint and a spatial coordinateof the image element of the first image and a distance between theviewpoint and a spatial coordinate of the at least one image element ofthe second image, and wherein the visibility factor is equal to zerowhen the at least one image element of the second image is closer thanthe image element of the first image to the viewpoint, and where thevisibility factor is equal to one when the image element of the firstimage is closer than the at least one image element of the second imageto the viewpoint, and where the visibility factor follows a gradualtransition between the values of zero and one.
 2. The method accordingto claim 1 where the spatial filter comprises a band-pass filter.
 3. Themethod according to claim 2 where the band-pass filter is selected froma group consisting of a high-pass filter, a low-pass filter and acombination of a high-pass filter and a low-pass filter.
 4. The methodaccording to claim 1 where the image element of the first image ismapped to a number of the image elements of the second image and wherethe strength of the spatial filter determines the value of said number.5. The method according to claim 4 where the image element of the firstimage and the number of image elements of the second image are alignedalong a horizontal line of the first image.
 6. The method according toclaim 1 where the strength of the spatial filter is a function of adifference between the depth of the image element of the first image andthe reference depth of the first image.
 7. The method according to claim1 where the depth of the at least one image element of the second imageis updated from an initial depth to a value between the initial depthand the depth of the image element of the first image, so that thesampling of the second image is done using the updated depth of the atleast one image element of the second image.
 8. The method according toclaim 1 where the first image signal has a 2.5D video image format. 9.The method of claim 1 where the reference depth represents a depth atwhich the first image is in focus.
 10. A multi-view display devicecomprising: a display panel comprising an array of display elements, thedisplay elements being arranged in groups, each group being associatedwith a different respective view direction (φ) of an image; and anoptical element for directing light emitted from the display panel sothat light emitted from each group of display elements is directed intoan angular distribution associated with the view direction of the group;said multi-view display device rendering image data for effecting theproduction of subimages by: receiving a first image signal representinga first image comprising 3D image data not yet rendered for multi-viewdisplay; spatially filtering the first image signal to provide a secondimage signal, the second image signal representing a second image, thespatial filtering comprising mapping an image element of the first imageto at least one image element of the second image, where a strength ofthe spatial filter is dependent on a depth of the image element of thefirst image relative to a reference depth of the first image; samplingthe second image signal to define a plurality of said sub-images, eachsub-image being associated with one of said a different respective viewdirections of the image; and displaying said plurality of saidsub-images on the display panel, wherein a visibility factor is appliedto the at least one image element of the second image, the visibilityfactor being determined from a distance between a viewpoint and aspatial coordinate of the image element of the first image and adistance between the viewpoint and a spatial coordinate of the at leastone image element of the second image, and wherein the visibility factoris equal to zero when the at least one image element of the second imageis closer than the image element of the first image to the viewpoint,and where the visibility factor is equal to one when the image elementof the first image is closer than the at least one image element of thesecond image to the viewpoint, and where the visibility factor follows agradual transition between the values of zero and one.
 11. A signalrendering system for rendering image data for effecting the productionof sub-images by a multi-view display, said system comprising at leastone processor including: an input module for receiving a first imagesignal representing a first image comprising 3D image data not yetrendered for multi-view display; a rendering module for spatiallyfiltering the first image signal to provide a second image signal, thesecond image signal representing a second image, the spatial filteringcomprising mapping an image element of the first image to at least oneimage element of the second image, where a strength of the spatialfilter is dependent on a depth of the image element of the first imagerelative to a reference depth of the first image; an output module forsampling the second image signal to define a plurality of saidsub-images, each sub-image being associated with a different respectiveview direction of the image; and a display module for displaying saidplurality of said sub-images on the multi-view display, wherein avisibility factor is applied to the at least one image element of thesecond image, the visibility factor being determined from a distancebetween a viewpoint and a spatial coordinate of the image element of thefirst image and a distance between the viewpoint and a spatialcoordinate of the at least one image element of the second image, andwherein the visibility factor is equal to zero when the at least oneimage element of the second image is closer than the image element ofthe first image to the viewpoint, and where the visibility factor isequal to one when the image element of the first image is closer thanthe at least one image element of the second image to the viewpoint, andwhere the visibility factor follows a gradual transition between thevalues of zero and one.
 12. A computer program embodied in anon-transitory computer readable medium for effecting the performance ofa method of rendering image data for producing sub-images on amulti-view display, the method comprising the steps of: receiving afirst image signal representing a first image comprising 3D image datanot yet rendered for multi-view display; spatially filtering the firstimage signal to provide a second image signal, the second image signalrepresenting a second image, the spatial filtering comprising mapping ofan image element of the first image to at least one image element of thesecond image, where a strength of the spatial filter is dependent on adepth of the image element of the first image relative to a referencedepth of the first image; sampling the second image signal to define aplurality of said sub-images, each sub-image being associated with adifferent respective view direction of the image; and displaying saidplurality of said sub-images on the multi-view display, wherein avisibility factor is applied to the at least one image element of thesecond image, the visibility factor being determined from a distancebetween a viewpoint and a spatial coordinate of the image element of thefirst image and a distance between the viewpoint and a spatialcoordinate of the at least one image element of the second image, andwherein the visibility factor is equal to zero when the at least oneimage element of the second image is closer than the image element ofthe first image to the viewpoint, and where the visibility factor isequal to one when the image element of the first image is closer thanthe at least one image element of the second image to the viewpoint, andwhere the visibility factor follows a gradual transition between thevalues of zero and one.